Technological Field
The present disclosure relates to development of parallel networks (e.g., computerized neural networks).
Background
Various approaches to development of non-spiking and/or spiking networks for computational purposes exist. When applied to computing and/or data processing, such networks may offer benefits of power efficiency, low latency, node failure resiliency, and are described easily. However, in some realizations these networks may not entirely be self-sufficient but rather operate jointly with programming modules developed using a conventionally specified computing approach. Such a module may deal with initializing the network, performing computations unsuited and/or offloaded from the network, interfacing with the user, storage and/or peripherals, converting input or output for the network, and interacting with other conventional components such as the operating system. As used hereinafter, the two computing components may be referred to as the “network” and the “conventional specification,” respectively.
The conventional component may typically be specified using a common purpose imperative programming language (e.g., C, Java®, Python®, and/or other) as a list of procedures which when executed may perform the desired operation. The common purpose languages are supported by a wide variety of integrated development environment tools (IDE) such as Microsoft Visual Studio®, Eclipse, GNAT Programming Studio, CodeWarrior, NetBeans, JBuilder®, and others. Such tools are capable of translating programming code of the common programming languages into machine-specific realization. Existing optimization techniques, such as peephole optimization, loop optimization, constant folding, register allocation, instruction scheduling, may be applied in order to adapt to particular computational hardware (e.g., RISC CPU, CISC CPU, DSP, FPGA, and/or other platforms).
The network component may often be more conveniently expressed by describing behavior of the network elements and/or how these elements are connected (e.g., one to all, all-to-all). For example, a winner-take-all spiking network may be described by specifying behavior of linear stochastic neurons and inhibitory synapses interconnecting neurons. While some implementations exist that may be capable of combining general-purpose language code with the description of parallel networks, implementations of the prior art do not provide a single unified approach capable of producing a hardware efficient implementation of neuron networks. Some existing realizations may utilize syntactically distinct specifications for the network elements (Brain, HLND). Specifically, this includes the behavior of the network elements specified in a syntax that is distinct, and incompatible with, the general-purpose language. If the network element specification is contained inside source files for the general purpose language, it is treated as a “string” and quoted so that the general purpose compiler does not attempt to parse it. Alternatively, other solutions do not allow the user to specify the behavior of network elements without implementing the hardware-specific version explicitly (such as NEST). In all of these prior solutions the user must describe their desired network behavior in a specialized syntactic description, distinct from the general-purpose language.